[WIP] ansible: Edit select-compiler.sh to select different versions of xcode #1967

AshCripps · 2019-10-18T12:39:16Z

Edited select-compiler.sh to change the xcode version used based on which version of node is being built.

Works on my local machine but needs to be checked to ensure the paths are correct on the build machines.

Will also require sudoers being edited to allow for xcode-select to run with sudo permissions.

Discussion here - #1964

mhdawson · 2019-10-18T13:25:19Z

jenkins/scripts/select-compiler.sh

+    sudo xcode-select -s /Applications/Xcode.app/Contents/Developer
+    echo "set xcode to latest - At least version 10"
+  elif [ "$NODEJS_MAJOR_VERSION" -ge "12" ]; then
+    sudo xcode-select -s /Applications/Xcode.app/Contents/Developer


How does this select a different version? /Applications/Xcode.app/Contents/Developer looks identical to the section for 13?

Atm it is identical becauase 12 and 13 both build on 10 +. The actual min. xcode doesnt change to version 10 until node 13 (according to BUILDING.md) so the question is do we want to use latest for these versions or hardcode in a stable version and only change it if required (security patches etc.).

For example /Applications/Xcode.11.1.app/Contents/Developer for node 13 and then something like /Applications/Xcode10.x.app/Contents/Developer for node 12.

Depends if we want as few xcodes on the machine as possible or if we want a stable xcode for each node release?

stable for each release, decided at time of first .0.0 for that release line, never changed afterwards.

Are these documented anywhere as to which version is used for each release?

in the BUILDING.md on the relevant release branch of nodejs/node, if documented, if not.... #1522 :-(

We're locked in to 8 for Node 12. We've historically locked in for the entirety of a release line. That could change and it would be lower risk on macOS than others, but not zero risk and the TSC would have to sign off on it and Build WG would shoulder the blame if things break. So .. for now consider it locked to Xcode 8 for Node 12.

sam-github · 2019-10-18T14:53:26Z

jenkins/scripts/select-compiler.sh

+    echo "set xcode to latest - At least version 10"
+  elif [ "$NODEJS_MAJOR_VERSION" -ge "12" ]; then
+    sudo xcode-select -s /Applications/Xcode.app/Contents/Developer
+    echo "set xcode to latest - at least version 8"


I think prev line should have been 8

sam-github · 2019-10-18T15:21:36Z

@AshCripps Would be good to add:

updates to https://github.com/nodejs/build/blob/master/ansible/MANUAL_STEPS.md#macos to describe how the sudoers needs to be setup
setup... how we will get the xcodes on the machine. There is a setup script: ./ansible/roles/package-upgrade/files/install-xcode.sh , can it do multiple versions?

AshCripps · 2019-10-18T15:26:31Z

2. setup... how we will get the xcodes on the machine. There is a setup script: ./ansible/roles/package-upgrade/files/install-xcode.sh , can it do multiple versions?

ATM it currently does 2 different versions based on what version of macos is on the box. So it should be easy enough to change it ive already tracked down where URLs for the different versions are so should be a simple change to install multiple versions

AshCripps · 2019-10-21T10:43:39Z

build/ansible/roles/package-upgrade/files/install-xcode.sh

Lines 7 to 15 in 9735105

    
           if [[ "$osx_vers" -ge 9 ]]; then 
        
           touch "$cmd_line_tools_temp_file"; 
        
           PROD=$(softwareupdate -l | 
        
             grep "\*.*Command Line" | 
        
             head -n 1 | awk -F"*" '{print $2}' | 
        
             sed -e 's/^ *//' | 
        
           tr -d '\n') 
        
           softwareupdate -i "$PROD"; 
        
           fi

The above lines don't work on the new nearform machines so the install-xcode script will have to be reworked to install xcode on the newer machines.

Also I found this blog post https://derflounder.wordpress.com/2018/06/10/updated-xcode-command-line-tools-installer-script-now-available/ which looks remarkably similar to what we have and the original file predates install-xcode.sh by about 3 years but the repo is under the MIT license but I don't see any link to the author or repo?

AshCripps · 2019-10-29T11:15:42Z

Ive edited the install-xcode script to allow for the installing of xcode - there are some problems here

Firstly the xcodes will need to be hosted on a server somewhere - the apple site requires a login to download the files and this is tricky on command line
Secondly it seems the xip files have certificates that expire as some old downloads I have from the other week I can no longer unzip
These are full blown xcodes on the new oses - this is required for notorization but also makes xcode-select easier IMO but they are big files ~10GB

Ive also edited the MANUALSTEPS.md to include the sudo line needed to run xcode-select

Next step is to sort out the select compiler script to change xcodes based on the OS version and what version of node is being built

rvagg · 2019-10-29T11:48:51Z

We have a spot on ci.nodejs.org where we put files required by our infra. See aix61-standalone/manualBootstrap.md for some examples of using https://ci.nodejs.org/downloads/. I don't see anything in Ansible but there was a time when I was doing ARMv7 compiler stuff on there too and I see the files are still there.
There might be problems with making these publicly available as there are probably redistribution restrictions on the files. Can you see anything associated with the files that says anything about redistribution? I don't have any great ideas for limiting distribution without opening up whole new security holes or setting up new dedicated resources.

rvagg · 2019-10-29T11:50:32Z

the xip files have certificates that expire

Oh, what? so even if we stored these files they will stop being extractable? So we're going to have to put ourselves at the mercy of Apple continue to supply these files?

What's the structure inside them? Is it practical to unpack and pack them up again into a format that we can properly archive for ourselves?

AshCripps · 2019-10-29T11:57:08Z

What's the structure inside them? Is it practical to unpack and pack them up again into a format that we can properly archive for ourselves?

Not sure, when you extract them you got a .app file which is actually a directory I believe as the xcode path becomes /Applications/Xcode.10.3.app/Contents/Developer

Ill have to try packing it up myself and see if someone else can reuse it.

AshCripps · 2019-10-29T16:55:18Z

Ill have to try packing it up myself and see if someone else can reuse it.

Tried this and it didnt work - showed an not from apple warning on both 10.14 and 10.15, you could work around it with a cmd-click but not sure this works on command line

AshCripps · 2020-02-27T13:36:45Z

Closing this, dont think this will be needed as notorization has forced us to update the release machines to new OS

AshCripps mentioned this pull request Oct 18, 2019

Supporting multiple versions of xcode #1964

Closed

mhdawson reviewed Oct 18, 2019

View reviewed changes

sam-github reviewed Oct 18, 2019

View reviewed changes

rvagg mentioned this pull request Oct 20, 2019

Revert "build: update minimum Xcode version for macOS" nodejs/node#30043

Closed

AshCripps force-pushed the macos-select-compiler branch from 8b59977 to 37ab522 Compare October 21, 2019 11:03

AshCripps added 5 commits October 29, 2019 10:27

ansible: Edit select-compiler.sh to select different versions of xcode

83681f1

Remove unneeded commented line

21d4cdb

clean up whitespace

eee452c

Set node 12 to use version 8

81c0518

Edit install script to install more xcodes + edit manual steps

473007b

AshCripps force-pushed the macos-select-compiler branch from 37ab522 to 473007b Compare October 29, 2019 10:56

AshCripps closed this Feb 27, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] ansible: Edit select-compiler.sh to select different versions of xcode #1967

[WIP] ansible: Edit select-compiler.sh to select different versions of xcode #1967

AshCripps commented Oct 18, 2019

mhdawson Oct 18, 2019

AshCripps Oct 18, 2019

sam-github Oct 18, 2019

AshCripps Oct 18, 2019

sam-github Oct 18, 2019

rvagg Oct 20, 2019

sam-github Oct 18, 2019

sam-github commented Oct 18, 2019

AshCripps commented Oct 18, 2019 •

edited

Loading

AshCripps commented Oct 21, 2019

AshCripps commented Oct 29, 2019

rvagg commented Oct 29, 2019

rvagg commented Oct 29, 2019

AshCripps commented Oct 29, 2019

AshCripps commented Oct 29, 2019

AshCripps commented Feb 27, 2020

[WIP] ansible: Edit select-compiler.sh to select different versions of xcode #1967

[WIP] ansible: Edit select-compiler.sh to select different versions of xcode #1967

Conversation

AshCripps commented Oct 18, 2019

mhdawson Oct 18, 2019

Choose a reason for hiding this comment

AshCripps Oct 18, 2019

Choose a reason for hiding this comment

sam-github Oct 18, 2019

Choose a reason for hiding this comment

AshCripps Oct 18, 2019

Choose a reason for hiding this comment

sam-github Oct 18, 2019

Choose a reason for hiding this comment

rvagg Oct 20, 2019

Choose a reason for hiding this comment

sam-github Oct 18, 2019

Choose a reason for hiding this comment

sam-github commented Oct 18, 2019

AshCripps commented Oct 18, 2019 • edited Loading

AshCripps commented Oct 21, 2019

AshCripps commented Oct 29, 2019

rvagg commented Oct 29, 2019

rvagg commented Oct 29, 2019

AshCripps commented Oct 29, 2019

AshCripps commented Oct 29, 2019

AshCripps commented Feb 27, 2020

AshCripps commented Oct 18, 2019 •

edited

Loading